Optimizing Resource Provisioning in Shared Cloud Systems

Cloud computing promises flexibility and high performance for users and cost efficiency for operators. To achieve this premise, cloud providers offer several provisioning strategies including long-term reserved resources and short-term ondemand resources. Determining the most appropriate provisioning strategy is a complex, multi-dimensional problem that depends on the load fluctuation, interference sensitivity and duration of incoming jobs and the performance unpredictability and cost of the provisioned resources. We first compare the two main provisioning strategies (reserved and on-demand resources) on Google Compute Engine (GCE) using three representative workload scenarios with mixes of batch and latency-critical applications and increasing levels of load variability. We show that either approach is suboptimal from the performance or cost perspective. We then explore hybrid provisioning strategies with both reserved and on-demand resources. We design policies that account for the resource preferences of incoming jobs to automatically determine which jobs should be mapped to reserved versus on-demand resources based on overall load, and resource unpredictability. We demonstrate that hybrid configurations improve both performance and cost-efficiency compared to fully reserved and fully on-demand systems. Specifically they improve performance by 2.1x compared to fully on-demand provisioning, and reduce cost by 46% compared to fully reserved systems. We also show that hybrid strategies are robust to variation in system and job parameters, such as cost, and system load.